Machine learning systems for making decisions about digital media are becoming more common. For example, machine-learned image models are increasingly being used in applications such as facial recognition, text and speech processing, computer-aided medical diagnosis, autonomous vehicle control, among other areas. Traditionally, the development of machine learning models is a time-consuming and error-prone process. The process typically involves a host of labor-intensive tasks, such as image annotation, that are performed manually by humans. The training process itself may require close supervision by data scientists over the course of the training, which may in some cases last days. Moreover, once training is completed, it is often difficult for a novice user to diagnose problems with the resulting model and determine corrective actions to improve model performance. The machine learning community currently lacks holistic systems for systematically developing machine-learned media models. Current tools require coding and are mostly single-user systems; they do not collaboration among various actors in a model building process such as data scientists, engineers, analysts, and product managers. There is a general need in the field for easy-to-use model development systems for rapidly developing machine-learned media models of high quality with minimal human dependency.
While embodiments are described herein by way of example for several embodiments and illustrative drawings, those skilled in the art will recognize that embodiments are not limited to the embodiments or drawings described. It should be understood, that the drawings and detailed description thereto are not intended to limit embodiments to the particular form disclosed, but on the contrary, the intention is to cover all modifications, equivalents and alternatives falling within the spirit and scope as defined by the appended claims. The headings used herein are for organizational purposes only and are not meant to be used to limit the scope of the description or the claims. As used throughout this application, the word “may” is used in a permissive sense (i.e., meaning having the potential to), rather than the mandatory sense (i.e., meaning must). Similarly, the words “include,” “including,” and “includes” mean including, but not limited to.
It will also be understood that, although the terms first, second, etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, a first contact could be termed a second contact, and, similarly, a second contact could be termed a first contact, without departing from the scope of the present invention. The first contact and the second contact are both contacts, but they are not the same contact.